1. Field of the Invention
The present invention relates to an image sensing apparatus and a correction method, and more specifically to a technique for correcting, in an image sensing apparatus that uses an image sensor, noise generated by the image sensor and the output of defective pixels in the image sensor.
2. Description of the Related Art
Heretofore, image sensing apparatuses such as digital cameras and video cameras that use an image sensor such as a CCD or a CMOS have become widespread. The pixel count of image sensors has increased with the improved performance of digital cameras, particularly in recent years.
While it is known that dark current occurs with these image sensors due to factors such as temperature, this dark current varies depending on the use environment, exposure period and the like. Therefore, a technique is known in which image capture is performed with the image sensor shielded just before or just after the actual image capture operation, and the obtained image (optical black image) is subtracted from the captured image (e.g., see “Description of the Related Art” in Japanese Patent Laid-Open No. 2003-333435). By thus subtracting the optical black image from the image of the object, image quality is improved through reducing the effects of tiny defects and fixed pattern noise in the dark current component.
A method is also disclosed in which a gain is applied so as to make fixed pattern noise stored after being extracted from an optical black image during the manufacture of a solid-state image sensor responsive to temperature change measured by a temperature sensor in the image sensing apparatus (e.g., see Japanese Patent Laid-Open No. 2003-018475). With this method, adverse effects resulting from the temperature dependency of fixed pattern noise is eliminated as a result of performing on the captured object image correction by subtracting fixed pattern noise to which a gain has been applied according to the temperature change.
The correction process disclosed in Japanese Patent Laid-Open No. 2003-333435 involving the subtraction of an optical black image enables the effects of fixed pattern noise and two-dimensional dark current unevenness on the screen, etc. to be reduced. However, the optical black image is taken by performing image capture with the image sensor shielded, for the same charge accumulation period as the actual image capture operation. Therefore, the shutter release time lag when capturing an image is increased, and the interval between first and second frames when performing continuous image capture is increased by the time taken to obtain the optical black image. Image sensing opportunities may be missed particularly at slow shutter speeds.
The method disclosed in Japanese Patent Laid-Open No. 2003-018475 seeks to eliminate the effects of temperature change in an image sensing apparatus, correction cannot be performed with high accuracy since it is not the temperature of the actual image sensor that is being measured and the temperature distribution on the screen varies depending on the image sensor.
With solid-state image sensors, on the other hand, defective pixels that occur in the manufacturing process are known to be one of the factors that degrade image quality and reduce manufacturing yield. Given the difficulty in completely eliminating defective pixels, it is generally known that image quality is improved by performing interpolation using pixels neighboring the defective pixels.
As for techniques that correct signals output from defective pixels, a method disclosed in the “Description of the Related Art” of Japanese Patent Laid-Open No. 2003-333435, for example, is known. This method firstly involves determining defective pixels using output values obtained by exposing the solid-state image sensor for a standard charge accumulation period under prescribed conditions when the solid-state image sensor is shipped from the factory, for instance. Information on output levels, positions and the like of defective pixels acquired at this time is stored, and when image capture is performed the output of the defective pixels is interpolated using the output levels of pixels neighboring the defective pixels, based on the stored information on output levels, positions and the like of defective pixels.
Further, techniques such as the following have been proposed for when shooting at slow shutter speeds. Firstly, an optical black image is taken prior to the actual image capture, and output levels at or above a prescribed output are extracted as defective pixels from the optical black image and stored. The extracted defective pixels are then corrected with respect to the image obtained by the actual image capture. So-called black thinning is performed on pixels other than the extracted defective pixels to subtract the output levels of pixels in the optical black image from the output levels of corresponding pixels obtained by the actual image capture. This enables correction to be performed that minimizes image degradation without the system breaking down, even with respect to an increase in the output levels of defective pixels that occurs when shooting at a slow shutter speed (e.g., see Japanese Patent Laid-Open No. 2001-028713).
A technique has also been proposed that involves providing plural pieces of correction data for correcting the output of defective pixels, and correcting the captured image after selecting using the optimal data selected in accordance with the shooting conditions such as sensitivity and charge accumulation period and the shooting environment such as temperature (e.g., see Japanese Patent Laid-Open No. 2003-333435).
Output levels of some defective pixels are caused by dark current and their output level varies greatly depending on the temperature and charge accumulation period. The output levels of defective pixels caused by dark current increase with the increase in the dark current due to the higher temperature and the longer exposure (the slower shutter speed).
A defective pixel affected by dark current described above causes the following phenomenon when charge accumulated in the defective pixel through an increase in dark current reaches the saturation level of the pixel. That is, a few percent of the output exceeding the saturation level of the pixel leaks into adjacent pixels. As a result, the output levels of the adjacent pixels become unnecessarily high, forming a cross-shaped defective pixel region, as shown in FIG. 10. In FIG. 10, the coordinates (n,n) indicate a defective pixel caused by dark current, and the increase in the output levels of pixels positioned at laterally and vertically adjacent thereto, that is, coordinates (n−1,n), (n+1,n) (n,n−1) and (n,n+1) as a result of charge leakage is shown.
With the method disclosed in the above Japanese Patent Laid-Open No. 2001-028713, the shutter release time lag from image sensing being instructed to image sensing actually being performed is lengthened, since time is needed prior to the actual image sensing to obtain an optical black image with the same charge accumulation period as the actual image sensing in order to extract defects. Particularly with shutter speeds slow enough for charge to leak into adjacent pixels, the shutter release time lag becomes extremely long. However, if attempts are made to shorten the shutter release time lag, it then becomes impossible to extract adjacent pixels into which charge has leaked, and correction cannot be performed.
Also, the following problems arise when attempting to perform correction such as disclosed in Japanese Patent Laid-Open No. 2003-333435 on a cross-shaped defective pixel region caused by dark current as shown in FIG. 10. That is, because output levels change depending on the temperature and charge accumulation period, information on whether or not to perform correction of adjacent pixels and addresses of the defective pixels need to be held for every condition with respect to each defective pixel, and the memory capacity for storing this data is required.
Also, in recent years, a proposal has been made to calculate saturated output using signals that have leaked into the floating diffusion (FD) portion of a CMOS sensor, and expand the dynamic range (e.g., see Japanese Patent Laid-Open No. 2006-222762). However, this does not amount to means for preventing image quality degradation, since it is impossible to judge whether signals that have leaked into the FD portion are caused by saturation due to light from the object or due to noise signals such as the dark current of defective pixels.